Visual Processing by Adaptive Decorrelation

Visual processing provides organisms with behaviorally relevant information about what is where in their environments. Challenges including environmental variability, and limitations on processing time and capacity make this task diicult. A model is presented which suggests how the visual system copes with these challenges by both passive and active mechanisms which work to decorrelate system responses to events that demand diierent behavioral responses. Passive mechanisms decorrelate neural responses along signiicant feature dimensions in parallel across image space. While limited foveal and attentional resources are actively deployed to locate and recognize objects. It is proposed that attention acts to select those features and regions which can best resolve behavioral uncertainties. Operationally, limited resources are deployed to regions or feature dimensions for which competing interpretations make contradictory predictions. It is this information that can best decorrelate candidate response activations allowing eecient resolution of pattern location and recognition decisions. This treatment of visual processing as adap-tive decorrelation allows a uniied perspective on data and function. Vision provides organisms with useful information about what is where in their environments (Marr 1982). However, which information is useful depends on the behavioral decisions and uncertainties confronting each particular organism at each particular moment. The early visual system in primates is tuned by evolution and development to evaluate each retinotopic location along a number of simple stimulus dimensions like luminance, orientation, color, motion , and depth. An interesting aspect of the neurophysi-ological data on this early processing is that it seems that it is not typically the absolute values along each dimension that are coded. Instead, multi-scale spatial-temporal contrasts (or relative intensities) along stimulus dimensions appear to be what is registered through neural mechanisms of adaptation and lateral inhibition. One way to understand this processing is as a means A B C Figure 1: The scenic features shown in the Lenna test image in A are severely masked under reduced lighting coming from the upper right of the scene in B. As for ur own visual systems, lightness by integrated contrast measures shows insensitivityto wide range of overall lighting levels and gradients. of decorrelating the neural responses to similiar intensity values at nearby locations. The functional value of decorre-lation is apparent when one considers that an organism can only deliver diierent responses to diierent events when it possesses separate neural populations whose responses are diierent or decorrelated for those events. In other words, the ability to distinguish …